Vehicle camera calibration apparatus and method

ABSTRACT

A vehicle camera calibration apparatus and method are provided. The vehicle camera calibration apparatus includes a camera module configured to acquire an image representing a road from a plurality of cameras installed in a vehicle, an input/output module configured to receive, as an input, the acquired image from the camera module, or output a corrected image, a lane detection module configured to detect a lane and extract a feature point of the lane from an image received from the input/output module, and a camera correction module configured to estimate a new external parameter using a lane equation and a lane width based on initial camera information and external parameter information in the image received from the input/output module, and to correct the image.

TECHNICAL FIELD

One or more example embodiments relate to a vehicle camera calibrationapparatus and method, and more particularly, to a vehicle cameracalibration apparatus and method that may automatically calibrate anexternal parameter of a camera based on an image of a lane on a roadreceived from a plurality of cameras installed in a vehicle while thevehicle is moving.

BACKGROUND ART

Recently, vehicles employ cameras for various purposes, for example, ablack box, or a rear camera for parking. Also, a plurality of camerasare installed in a vehicle to provide a driver with an image appearingas if he or she looks down at the vehicle from the sky, and thus a topview system capable of completely eliminating blind spots of a frontside, a rear side, a left side and a right side of the vehicle is beingactively developed.

However, when a plurality of cameras are installed in a vehicle, imageinformation transmitted by each of the cameras needs to be integratedand for this, coordinate system is the most important. In other words,each of the images obtained is input together with a variety ofinformation, and real-time spatial information among other storedinformation of each of the captured images is changed to a preset commoncoordinate system of the vehicle. A plurality of captured images may bematched based on the real-time spatial information changed to the commoncoordinate system, making it possible to display a top view image to adriver.

However, it is practically impossible to completely match a plurality ofcamera coordinate axes to the common coordinate system of the vehicledue to an error that occurs when a camera is mounted in the vehicle.This is because although the error may be precisely calibrated in avehicle production factory, an installation error of a camera mounted ina vehicle continues to occur due to a change in a position of the cameraby a physical force, such as a twist or an impact during driving, afterthe vehicle is released. Thus, a plurality of cameras mounted in avehicle need to be calibrated periodically, or at every impact.

To match images input from a plurality of cameras after a vehicle isreleased, camera calibration needs to be performed to correct anyinstallation error of a camera as a preliminary consideration. For thecamera calibration, information such as installation height andinstallation angle of an installed camera, is required. In a relatedart, to acquire information for such a camera calibration, a method ofinstalling a specific reference pattern, such as a checkered board, onthe ground, capturing the specific reference pattern and acquiringinformation based on a pattern image is used.

It is common to oblation the camera information by using a specificreference pattern, a relative position of the specific marker is knownthis advance, precise camera information may be acquired. However, toperform the method, there is a need to perform an operation aftersecuring a space large enough to install a specific reference patternaround a vehicle.

In particular, since an installation error of a camera occursperiodically or every time an impact occurs due to actual travelling ofthe vehicle, it is too troublesome, time consuming, and costly for anoperator to learn camera information using a camera informationacquisition method using a specific reference pattern, and to perform acamera calibration every time the installation error occurs. Thus, thereis a need to automatically calibrate a camera even when a vehicle istravelling.

Korean Patent Publication No. 10-2008-0028531, published on Apr. 1,2008, is provided as a related art.

DISCLOSURE OF INVENTION Technical Subject

Example embodiments provide a vehicle camera calibration apparatus andmethod that may receive images representing a travelling road of avehicle from a plurality of cameras installed in the vehicle duringtravelling of the vehicle, that may detect a lane of the road from theimages and that may automatically correct an external parameter of acamera installed in the vehicle based on the detected lane, so as toquickly and easily calibrate a camera in which a physical installationerror occurs due to travelling of the vehicle, even when the vehicle istravelling.

However, aspects of example embodiments of the present disclosure arenot limited to those described above. Other aspects that are notdescribed herein will be clearly understood by a person having ordinaryskill in the art from descriptions below.

Technical Solution

According to an aspect, there is provided a vehicle camera calibrationapparatus including a camera module configured to acquire an imagerepresenting a road from a plurality of cameras installed in a vehicle,an input/output module configured to receive, as an input, the acquiredimage from the camera module, or output a corrected image, a lanedetection module configured to detect a lane and extract a feature pointof the lane from an image received from the input/output module, and acamera correction module configured to calculate a lane equation and alane width based on initial camera information and external parameterinformation in the image received from the input/output module, toestimate a new external parameter, and to correct the image.

The input/output module may include an image inputter/outputterconfigured to receive, as an input, an image from the camera module orto output a corrected image to another module, and a storage configuredto receive the corrected image and changed external parameterinformation from the camera correction module, and to store thecorrected image and the changed external parameter information.

The lane detection module may include an image processor configured toremove noise from the image received from the input/output module, aframe accumulator configured to accumulate images processed by the imageprocessor as consecutive frames, when a lane is marked with a dashedline, an edge detector configured to detect an edge and to extract afeature point from a cumulative image including the images accumulatedby the frame accumulator, and a lane determiner configured to determinea lane from the feature point extracted by the edge detector.

The camera correction module may include an image converter configuredto extract a feature point of a lane from a received image based oninitial camera information and external parameter information and toconvert the received image into a top-view image, an equation calculatorconfigured to acquire a lane equation from the top-view image acquiredby the image converter, a parameter estimator configured to estimate anew external parameter using the lane equation calculated by theequation calculator, and a corrector configured to correct an imageusing the new external parameter estimated by the parameter estimator.

According to another aspect, there is provided a vehicle cameracalibration method including a lane detection operation of receiving, asan input, an acquired image from a camera module and detecting a lane,an image conversion operation of extracting a feature point of a lanefrom a received image based on initial camera information and externalparameter information, and converting the received image into a top-viewimage, an equation calculation operation of acquiring a lane equationand a lane width from the top-view image acquired in the imageconversion operation, a parameter estimation operation of estimating anew external parameter using the lane equation calculated in theequation calculation operation, and an image correction operation ofcorrecting an image using the new external parameter estimated in theparameter estimation operation.

The lane detection operation may include an image processing operationof removing noise from an image received from an input/output module, aframe accumulation operation of accumulating images processed in theimage processing operation as consecutive frames when a lane is markedwith a dashed line in an image processed in the image processingoperation, and immediately performing a next operation when the lane ismarked with a straight line, an edge detection operation of detecting anedge and extracting a feature point from a cumulative image includingthe images accumulated in the frame accumulation operation, and a lanedetermination operation of determining a lane from the feature pointextracted in the edge detection operation.

The image processing operation may include performing an algorithm ofsoftening a boundary of an object in an image to remove noise.

The image processing operation may include removing noise by changing asize of an image to prevent a region corresponding to a split lane in anacquired image from being detected as an edge.

The frame accumulation operation may include comparing all pixel valuesof the cumulative image and storing a highest pixel value in a new imageso that a dashed-line lane in an acquired image is viewed to beidentical to a solid-line lane.

The edge detection operation may include detecting an edge using analgorithm and extracting the edge and central coordinates of the edge asfeature points.

The lane determination operation may include detecting a longest outlineamong lines represented by feature points extracted in the edgedetection operation and determining the longest outline as a lane.

The lane equation calculated in the equation calculation operation maybe expressed as an equation of a straight line, or a multi-dimensionalequation.

The parameter estimation operation may include a primary correctionoperation of determining whether correction information of a pitch (Rx),a yaw (Ry) and a roll (Rz) of a front camera of a vehicle, andcorrection information of a pitch (Rx), a yaw (Ry) and a roll (Rz) of arear camera of the vehicle are accurate, of performing a next operationwhen the correction information is accurate, and of calculating thecorrection information of the pitch (Rx), the yaw (Ry) and the roll (Rz)of the front camera and the correction information of the pitch (Rx),the yaw (Ry) and the roll (Rz) of the rear camera and returning to theimage conversion operation when the correction information isinaccurate, a secondary correction operation of determining whethercorrection information of a position translation (Tz) of a rear cameraof a vehicle is accurate, of performing a next operation when thecorrection information is accurate, and of calculating the correctioninformation of the position translation (Tz) of the rear camera andreturning to the image conversion operation when the correctioninformation is inaccurate, tertiary correction operation of determiningwhether correction information of a pitch (Rx), a yaw (Ry), a roll (Rz)and a position translation (Tz) of each of a left camera and a rightcamera of a vehicle is accurate, of performing a next operation when thecorrection information is accurate, and of calculating the correctioninformation of the pitch (Rx), the yaw (Ry), the roll (Rz) and theposition translation (Tz) of each of the left camera and the rightcamera and returning to the image conversion operation when thecorrection information is inaccurate, and a storage operation of storingexternal parameters calculated in the primary correction operationthrough the tertiary correction operation.

The correction information of the pitch (Rx) of each of the front cameraand the rear camera in the primary correction operation may becalculated by modifying a new pitch (Rx) and repeating a process fromthe image conversion operation until Equation 1 shown below issatisfied.

$\begin{matrix}{{\frac{1}{G_{l}} - \frac{1}{G_{r}}} = 0} & \left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack\end{matrix}$

Here, G_(l) denotes a slope of a left lane, and G_(r) denotes a slope ofa right lane.

The correction information of the yaw (Ry) of each of the front cameraand the rear camera in the primary correction operation and thesecondary correction operation may be calculated by modifying a new yaw(Ry) and repeating a process from the image conversion operation untilEquation 2 shown below is satisfied.

W _(l) −W _(r)=0  [Equation 2]

Here, W_(l) denotes a width of a left lane, and W_(r) denotes a width ofa right lane.

The correction information of the roll (Rz) of each of the front cameraand the rear camera in the primary correction operation may becalculated by modifying a new roll (Rz) and repeating a process from theimage conversion operation until Equation 3 shown below is satisfied.

$\begin{matrix}{{\frac{1}{G_{l}} + \frac{1}{G_{r}}} = 0} & \left\lbrack {{Equation}\mspace{14mu} 3} \right\rbrack\end{matrix}$

Here, G_(l) denotes a slope of a left lane, and G_(r) denotes a slope ofa right lane.

The correction information of the position translation (Tz) of the rearcamera in the secondary correction operation may be calculated bymodifying a new position translation (Tz) and repeating a process fromthe image conversion operation until Equation 4 shown below issatisfied.

C _(d)−_(r)=0  [Equation 4]

Here, C_(f) denotes a width between a left lane and a right lane in afront road, and C_(r) denotes a width between a left lane and a rightlane in a rear road.

The correction information of the pitch (Rx) of the left camera in thetertiary correction operation may be calculated by modifying a new pitch(Rx) and repeating a process from the image conversion operation untilEquation 5 shown below is satisfied. The correction information of thepitch (Rx) of the right camera in the tertiary correction operation maybe calculated by modifying a new pitch (Rx) and repeating a process fromthe image conversion operation until Equation 6 shown below issatisfied.

$\begin{matrix}{{\frac{\left( {x_{2} + x_{6}} \right)}{2} - \frac{\left( {x_{1} + x_{5}} \right)}{2}} = 0} & \left\lbrack {{Equation}\mspace{14mu} 5} \right\rbrack \\{{\frac{\left( {x_{3} + x_{7}} \right)}{2} - \frac{\left( {x_{4} + x_{8}} \right)}{2}} = 0} & \left\lbrack {{Equation}\mspace{14mu} 6} \right\rbrack\end{matrix}$

Here, x₁ denotes a left x-coordinate of a left lane of a front road, x₂denotes a right x-coordinate of the left lane of the front road, x₅denotes a left x-coordinate of a left lane of a rear road, x₆ denotes aright x-coordinate of the left lane of the rear road, x₃ denotes a leftx-coordinate of a right lane of the front road, x₄ denotes a rightx-coordinate of the right lane of the front road, x₇ denotes a leftx-coordinate of a right lane of the rear road, and x₈ denotes a rightx-coordinate of the right lane of the rear road.

The correction information of the yaw (Ry) of the left camera in thetertiary correction operation may be calculated by modifying a new yaw(Ry) and repeating a process from the image conversion operation untilEquation 7 shown below is satisfied. The correction information of theyaw (Ry) of the right camera in the tertiary correction operation may becalculated by modifying a new yaw (Ry) and repeating a process from theimage conversion operation until Equation 8 shown below is satisfied.

(x ₂₂ −x ₁₁)−(x ₆₆ −x ₅₅)=0[Equation 7]

(x ₄₄ −x ₃₃)−(x ₈₈ −x ₇₇)=0  [Equation 8]

Here, x₁₁ denotes a left x-coordinate of a left lane of a front road,x₂₂ denotes a right x-coordinate of the left lane of the front road, x₅₅denotes a left x-coordinate of a left lane of a rear road, x₆₆ denotes aright x-coordinate of the left lane of the rear road, x₃₃ denotes a leftx-coordinate of a right lane of the front road, x₄₄ denotes a rightx-coordinate of the right lane of the front road, x₇₇ denotes a leftx-coordinate of a right lane of the rear road, and x₈₈ denotes a rightx-coordinate of the right lane of the rear road.

The correction information of the roll (Rz) of the left camera in thetertiary correction operation may be calculated by modifying a new roll(Rz) and repeating a process from the image conversion operation untilEquation 9 shown below is satisfied. The correction information of theroll (Rz) of the right camera in the tertiary correction operation maybe calculated by modifying a new roll (Rz) and repeating a process fromthe image conversion operation until Equation 10 shown below issatisfied.

$\begin{matrix}{\frac{1}{G_{l}} = 0} & \left\lbrack {{Equation}\mspace{14mu} 9} \right\rbrack \\{\frac{1}{G_{r}} = 0} & \left\lbrack {{Equation}\mspace{14mu} 10} \right\rbrack\end{matrix}$

Here, G_(l) denotes a slope of a left lane, and G_(r) denotes a slope ofa right lane.

The correction information of the position translation (Tz) of the leftcamera in the tertiary correction operation may be calculated bymodifying a new position translation (Tz) and repeating a process fromthe image conversion operation until Equation 11 shown below issatisfied. The correction information of the position translation (Tz)of the right camera in the tertiary correction operation may becalculated by modifying a new position translation (Tz) and repeating aprocess from the image conversion operation until Equation 12 shownbelow is satisfied.

$\begin{matrix}{{\frac{\left( {x_{222} - x_{111}} \right) + \left( {x_{10} - x_{999}} \right)}{2} - \left( {x_{666} - x_{555}} \right)} = 0} & \left\lbrack {{Equation}\mspace{14mu} 11} \right\rbrack \\{{\frac{\left( {x_{444} - x_{333}} \right) + \left( {x_{12} - x_{11}} \right)}{2} - \left( {x_{888} - x_{777}} \right)} = 0} & \left\lbrack {{Equation}\mspace{14mu} 12} \right\rbrack\end{matrix}$

Here, x₁₁₁ denotes a left x-coordinate of a left lane of a front road,x₂₂₂ denotes a right x-coordinate of the left lane of the front road,x₅₅₅ denotes a left x-coordinate of a left lane, x₆₆₆ denotes a rightx-coordinate of the left lane, x₉₉₉ denotes a left x-coordinate of aleft lane of the rear road, x₁₀ denotes a right x-coordinate of the leftlane of the rear road, x₃₃₃ denotes a left x-coordinate of a right laneof a front road, x₄₄₄ denotes a right x-coordinate of a right lane of afront road, x₇₇₇ denotes a left x-coordinate of a right lane, x₈₈₈denotes a right x-coordinate of a right lane, x₁₁ denotes a leftx-coordinate of a right lane of the rear road, and x₁₂ denotes a rightx-coordinate of the right lane of the rear road.

Effect of the Invention

According to example embodiments, a camera calibration may be performedby receiving images representing a travelling road of a vehicle capturedfrom a plurality of cameras installed in the vehicle, by detecting alane from the images, and by correcting an external parameter of acamera installed in the vehicle based on the detected lane. Thus, it ispossible to quickly and simply calibrate a camera without the need tostop the vehicle.

Also, according to example embodiments, a camera calibration may beperformed during travelling of a vehicle, and thus there is no need toperform a secondary operation, such as an operation of securing a spaceand installing a correction pattern after a vehicle stops for acorrection of a camera installed in the vehicle in a related art. Thus,it is possible to reduce time or costs required for a camera correction.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating a vehicle camera calibration apparatusaccording to an example embodiment.

FIG. 2 is a flowchart illustrating a vehicle camera calibration methodaccording to an example embodiment.

FIG. 3 is a flowchart illustrating operations included in a lanedetection operation of FIG. 2.

FIG. 4 is a flowchart illustrating operations included in a parameterestimation operation of FIG. 2.

FIG. 5 illustrates an example in which pitches (Rx) of a front cameraand a rear camera are twisted in an image acquired from a camera moduleof FIG. 1.

FIG. 6 illustrates an example in which yaws (Ry) of a front camera and arear camera are twisted in an image acquired from the camera module ofFIG. 1.

FIG. 7 illustrates an example in which rolls (Rz) of a front camera anda rear camera are twisted in an image acquired from the camera module ofFIG. 1.

FIG. 8 illustrates an example in which a position translation (Tz) of arear camera is twisted in an image acquired from the camera module ofFIG. 1.

FIG. 9 illustrates an example in which pitches (Rx) of a left camera anda right camera are twisted in an image acquired from the camera moduleof FIG. 1.

FIG. 10 illustrates an example in which yaws (Ry) of a left camera and aright camera are twisted in an image acquired from the camera module ofFIG. 1.

FIG. 11 illustrates an example in which rolls (Rz) of a left camera anda right camera are twisted in an image acquired from the camera moduleof FIG. 1.

FIG. 12 illustrates an example in which position translations (Tz) of aleft camera and a right camera are twisted in an image acquired from thecamera module of FIG. 1.

BEST MODE FOR CARRYING OUT THE INVENTION

Advantages and features of example embodiments of the presentdisclosure, and methods of achieving the same will be clearly understoodwith reference to the accompanying drawings and the following detailedexample embodiments. However, the present disclosure may be embodied inmany different forms and should not be construed as being limited to theexample embodiments set forth herein. Rather, these example embodimentsare provided so that this disclosure will be thorough and complete, andwill fully convey the inventive concept to those of ordinary skill inthe art. The inventive concept is defined by the appended claims.Therefore, the meanings of terms should be interpreted based on thescope throughout this specification.

When it is determined detailed description related to a related knownfunction or configuration they may make the purpose of the presentinvention unnecessarily ambiguous in describing the present invention,the detailed description will be omitted here. Also, terminologies usedherein are defined to appropriately describe the embodiments of thepresent invention and thus may be changed depending on a user, theintent of an operator, or a custom of a field to which the presentinvention pertains. Accordingly, the terminologies must be defined basedon the following overall description of this specification.

Hereinafter, example embodiments of the present disclosure will bedescribed in detail with reference to the accompanying drawings.

To actually use a vehicle camera calibration apparatus and methodaccording to an example embodiment, a lane may need to be viewed withoutany obstacle in an image captured by a camera installed in a vehicle,and both a dashed-line lane and a solid-line lane may need to bepresent. Also, the road may need to be flat and straight.

For example, when a road is not flatland, a correction may be performed,however, the accuracy of a corrected external parameter may decrease. Toprevent the above problem, it is desirable to perform the correction ona flatland.

Also, an algorithm that will be described below is an example of thepresent disclosure, and accordingly the present disclosure is notlimited to the algorithm. For example, another algorithm that performs acorresponding function may be used.

FIG. 1 is a diagram illustrating a vehicle camera calibration apparatusaccording to an example embodiment. The vehicle camera calibrationapparatus may include a camera module 100 configured to acquire an imagerepresenting a road from a plurality of cameras installed in a vehicle,an input/output module 200 configured to receive, as an input, theacquired image from the camera module 100 or to output a correctedimage, a lane detection module 300 configured to detect a lane andextract a feature point of the lane from an image received from theinput/output module 200, and a camera correction module 400 configuredto calculate a lane equation and a lane width based on initial camerainformation and external parameter information in the image receivedfrom the input/output module 200, to estimate a new external parameter,and to correct the image.

The camera module 100 refers to a device in which a plurality of camerasare installed in an arbitrary position of the vehicle, and may perform afunction of acquiring an image by capturing a road from the plurality ofcameras. In an example embodiment, a total of four cameras are installedin a front side, a rear side, a left side and a right side of a vehicle.Thus, all surroundings of the vehicle may be visualized by eliminating ablind spot when the surroundings of the vehicle are captured. This ismerely an example, and a number of cameras installed in the vehicle maybe increased or reduced.

A camera of the camera module 100 may use a wide-angle lens with aviewing angle greater than that of a typical lens, or a fisheye lensthat is a super-wide-angle lens exceeding 180°. Accordingly, to generatea top-view image, it is desirable to capture all surroundings of avehicle during matching of camera images. However, a type of cameras ofthe camera module 100 capable of calibrating a vehicle camera may beapplicable regardless of the above-described angle of view.

When the four cameras of the camera module 100 are installed on thefront side, the rear side, the left side and the right side of thevehicle, and are calibrated using a lane, posture information of each ofthe cameras, that is, a pitch (Rx), a yaw (Ry) and a roll (Rz) that areexternal parameters indicating a degree of rotation of a camera at aspecific position, and a position translation (Tz) that is an externalparameter indicating a degree to which a rear camera, a left camera anda right camera are moved from specific positions, may be estimated.

In other words, in an example of a straight lane on a straight road, thestraight lane may appear to be vertical when viewed from a top-viewimage, and all lanes shown using the four cameras may be present on oneof straight lanes. In this example, when a camera correction isaccurate, the lanes may be viewed as one lane. When the cameracorrection is inaccurate, the lanes may be viewed as several lanes.Based on the above characteristic, it is possible to estimate severalexternal parameters.

However, it is assumed that when a correction is performed as describedabove, a vehicle travels on a straight lane on a flat ground and bothlanes and vehicles travel in parallel.

Also, the camera module 100 may be connected via a wire or wirelessly tothe input/output module 200 that will be described below, and maytransmit an image captured by the camera.

The input/output module 200 may be a module configured to receive, as aninput, an image acquired from the camera module 100, or to output acorrected image from the camera correction module 400 that will bedescribed below to an output device that may be recognized by a user.The input/output module 200 may include an image inputter/outputter 210,and a storage 220.

When an image captured by a camera of the camera module 100 is receivedand transferred to an image processor 310 of the lane detection module300, or when an image corrected by a corrector 440 of the cameracorrection module 400 is stored in the storage 220 that will bedescribed below, the image inputter/outputter 210 may receive the imageand output the image to an external device, such as a monitor, and thelike, that may be recognized by a user. Also, the imageinputter/outputter 210 may perform an image preprocessing process by animage filter, and the like, if necessary.

The storage 220 may store the image corrected by the corrector 440 ofthe camera correction module 400 as described above, and may also storeinformation about an external parameter of a new camera estimated by aparameter estimator 430 of the camera correction module 400.

Also, the storage 220 may store initial camera information, that is,information about an internal parameter, such as a focal length, aprincipal point, a distortion coefficient, and the like, correspondingto unique characteristics of a plurality of camera lenses of the cameramodule 100, and store initial external parameter information, such asinitial posture information and position information of a plurality ofcameras of the camera module 100.

The initial camera information and the initial external parameterinformation stored in the storage 220 may be provided to the cameracorrection module 400 that will be described below, and may be used toestimate a new external parameter. An image completely corrected in thecamera correction module 400 may be provided to be processed andutilized in various devices, for example, a top view system, a panoramicview system, and the like.

The lane detection module 300 may perform a function of detecting a laneand extracting a feature point of the lane from an image received fromthe input/output module 200, and may be a module that includes variousalgorithms to perform the function. The lane detection module 300 mayinclude the image processor 310, a frame accumulator 320, an edgedetector 330, and a lane determiner 340.

The image processor 310 may include an image processing algorithm toremove noise from an image received from the image inputter/outputter210 of the input/output module 200 and to perform image processing sothat a boundary of a lane on a road may be more clearly shown.

An algorithm used by the image processor 310 to remove noise from animage and to perform the image processing is not limited to an algorithmthat will be described below, and may be replaced with another algorithmto perform a corresponding function.

The image processor 310 may convert a color image received from theimage inputter/outputter 210 into a monochromatic image, for theabove-described noise removal and image processing. In other words, theimage processor 310 may use the received color image for the noiseremoval and the image processing without a change, or may convert thecolor image into a monochromatic image and use the monochromatic image.

The image processor 310 may apply an algorithm to remove noise from animage, as described above. For example, the image processor 310 mayperform a morphological operation and median filtering.

The morphological operation performed by the image processor 310 may bea type of image filtering used in a preprocessing process, for example,a noise removal, an extraction of a feature point, and the like, priorto a separation of images, that is, a real image processing, and mayinclude an erosion operation and a dilation operation. According to anexample embodiment, a morphological opening may be performed after amorphological closing.

A morphological closing performed in the image processor 310 may have aneffect of softening a boundary of an object appearing in an image, so asto remove discontinuous data from the image by softening the boundary.

By a morphological opening performed after the morphological closingperformed in the image processor 310, noise may be removed and a size ofan object may be maintained.

To remove noise, the image processor 310 may perform median filtering aswell as the morphological closing and the morphological opening that aresequentially applied.

The image processor 310 may remove noise by changing a size of an imageafter the median filtering is performed. Split lanes may appear in aconsiderable number of portions of images acquired by a left camera anda right camera of the camera module 100, and a lane recognition rate inthe lane determiner 340 may decrease when the portions corresponding tothe split lanes are detected as edges by the edge detector 330. Thus, itis possible to prevent a reduction in the lane recognition rate byremoving noise from the portions corresponding to the split lanes.

When a lane appearing in an image processed by the image processor 310,that is, an image from which noise is removed, is marked with a dashedline, the frame accumulator 320 may perform a function of accumulatingthe image as consecutive frames. When the lane is marked with a straightline, an image processed by the image processor 310 may be transmittedto the edge detector 330 by passing through the frame accumulator 320without a change.

In an example embodiment, the frame accumulator 320 may accumulate fourframes. A number of frames to be accumulated is merely an example, andmay be changed, if necessary.

Image frames may be accumulated in the frame accumulator 320 asdescribed above, so that a dashed-line lane appearing in an image may beviewed to be identical to a solid-line lane. This is because the lanedetector 340 more easily and accurately determine and detect asolid-line lane.

Through an accumulation of frames in the frame accumulator 320, adashed-line lane in an image may be viewed to be identical to asolid-line lane, by comparing all pixel values in four consecutiveimages from which noise is removed, that is, a cumulative imageincluding the four accumulated images, and by storing a highest pixelvalue in a new image.

The edge detector 330 may detect an edge and extract a feature pointfrom a lower end portion of a cumulative image of the frame accumulator320.

Since a lane appears in a lower half region of the cumulative image andsky appears in an upper region, the edge detector 330 may define aregion other than the above regions as a region of interest (ROI) toreduce a number of operations.

Since a boundary between a road color and a lane color is clear, theedge detector 330 may detect an edge using a Canny edge algorithm thatmay remove all edges associated with a gray matter of the original imagewhile most effectively finding a contour, after an edge is allowed tostrongly appear through an image processing. Due to a low error rate andaccurate positions of edge points, the Canny edge algorithm may be usedin the edge detector 330. Although the Canny edge algorithm is providedas an example of an edge detection algorithm used in the edge detector330, example embodiments are not limited thereto. Accordingly, otheralgorithms may also be used if necessary.

The edge detector 330 may extract the edge and central coordinates ofthe edge as feature points. Because an edge appears as noise of avehicle in an image as well as a lane in the image, an edge and centralcoordinates of the edge may be used to reduce an edge that appearswidely.

When the edge detector 330 extracts the edge and the central coordinatesof the edge as feature points, the lane in the image may be displayed asa single long line, and noise may be displayed as a relatively shortline or may not be displayed.

The lane determiner 340 may determine a lane from a feature pointextracted in the edge detector 330.

Various schemes of determining a lane from a feature point in the lanedeterminer 340 may be used. For example, the lane determiner 340 maydetect the longest outline among lines represented by feature pointsextracted in the edge detector 330, and may determine the longestoutline as a lane.

The lane determiner 340 may find two outlines from images acquired bythe front camera and the rear camera of the camera module 100, and mayfind one outline from images acquired by the left camera and the rightcamera. This is because a total of two lanes on a left side and a rightside are monitored by the front camera and the rear camera, and one laneis monitored by the left camera and the right camera. The lanedeterminer 340 may determine the found outlines as lanes.

The camera correction module 400 may be a module configured to perform afunction of correcting external parameters of the four cameras that areinstalled in the front side, the rear side, the left side and the rightside of the vehicle, based on a lane detected by the lane detectionmodule 300. The camera correction module 400 may include an imageconverter 410, an equation calculator 420, the parameter estimator 430,and the corrector 440.

The camera correction module 400 may calculate a lane equation and alane width based on initial camera information and external parameterinformation in an image received from the input/output module 200, mayestimate a new external parameter based on the calculated lane equationand the calculated lane width, and may correct the image.

When an external parameter is accurate and when a conversion to atop-view image is performed, lanes on the top-view image may be on thesame straight line and may be vertically shown. However, when a positionof a camera of the camera module 100 is changed by a physical force, anaccuracy of the external parameter may decrease and the lanes on thetop-view image may not exist on a straight line. Thus, based on theabove characteristic, an external parameter of a changed camera may beestimated.

The image converter 410 may extract a feature point of a lane from areceived image based on initial camera information and externalparameter information, and may convert the received image into atop-view image.

The image converter 410 may convert the received image into a top-viewimage, based on a vertically represented top-view image, however, thereis not limitation thereto. In addition to the vertically representedtop-view image, a top-view image at any angle may also be used.

For example, a top-view image that allows a vehicle to be horizontallyviewed as well as a top-view image that allows a vehicle to bevertically viewed may be configured. In this example, an equationassociated with a slope that will be described below may be defineddifferently based on a vertical top-view image or a horizontal top-viewimage.

In other words, when a top-view image that is converted so that avehicle is horizontally viewed is configured, a corresponding slope maybe perpendicular to a slope of a top-view image that is converted sothat a vehicle is vertically viewed, and accordingly all equations forthe slopes may be changed. However, based on a top-view image convertedso that a vehicle is vertically viewed, a corresponding slope and anequation may be determined, which will be described below. This ismerely an example, and a top-view image that is converted so that avehicle is horizontally viewed may also be used.

The equation calculator 420 may acquire the lane equation by linefitting of feature points in the top-view image acquired in the imageconverter 410. For the line fitting, a least mean square (LMS) algorithmmay be used. Also, the lane equation may be an equation of a straightline, or a multi-dimensional equation.

The parameter estimator 430 may estimate a new external parameter basedon the lane equation calculated by the equation calculator 420.

The corrector 440 may correct an image based on the new externalparameter estimated by the parameter estimator 430, and may transmit theimage to the storage 220.

Hereinafter, a method of calibrating a vehicle camera using a vehiclecamera calibration apparatus configured as described above will bedescribed in detail.

FIG. 2 is a flowchart illustrating a vehicle camera calibration methodaccording to an example embodiment. The vehicle camera calibrationmethod may include lane detection operation S510, image conversionoperation S520, equation calculation operation S530, parameterestimation operation S540, and image correction operation S550.

Referring to FIG. 3, lane detection operation S510 may be an operationof detecting a lane and extracting a feature point of the lane in thelane detection module 300 when an image acquired from a camera of thecamera module 100 is input through the image input/output module 200,and may include image processing operation S511, frame accumulationoperation S512, edge detection operation S513, and lane determinationoperation S514.

Image processing operation S511 may be an operation of removing noisefrom a received image and of performing an image processing to moreclearly display a boundary of a lane of a road using the image processor310 in the lane detection module 300.

The image processor 310 may convert a color image received from theimage inputter/outputter 210 into a monochromatic image, for theabove-described noise removal and image processing. In other words, theimage processor 310 may use the received color image for the noiseremoval and the image processing without a change, or may convert thecolor image into a monochromatic image and use the monochromatic image.

In image processing operation S511, an algorithm of softening a boundaryof an object in an image to remove noise may be performed. For example,in an example embodiment, a morphological operation may be performed.Also, a median filtering may be performed to remove additional noise.

In image processing operation S511, noise may be removed by changing asize of an image on which median filtering is performed, to prevent aregion corresponding to a split lane in an acquired image from beingdetected as an edge.

Example embodiments are not limited to the above morphological operationalgorithm and the above median filtering algorithm performed in imageprocessing operation S511, and other algorithms that performcorresponding functions may be used.

In frame accumulation operation S512, the lane detection module 300 mayaccumulate, using the frame accumulator 320, an image from which noiseis removed as consecutive frames, when a lane is marked with a dashedline in an image processed in image processing operation S511. When thelane is marked with a straight line, the next operation may be performedimmediately.

In frame accumulation operation S512, when a lane appearing in anacquired image is marked with a dashed line, instead of a straight line,all pixel values in a cumulative image may be compared so that adashed-line lane is viewed to be identical to a solid-line lane that isa straight line, and a highest pixel value may be stored in a new image.

Edge detection operation S513 may refer to an operation of detecting anedge and extracting a feature point from a lower end portion of thecumulative image using the edge detector 330 of the lane detectionmodule 300.

In edge detection operation S513, an edge may be detected, and the edgeand central coordinates of the edge may be extracted as feature points.For example, a Canny edge algorithm may be used.

Lane determination operation S514 may refer to an operation ofdetermining a lane from a feature point extracted in the edge detector330 using the lane determiner 340 of the lane detection module 300.

In lane determination operation S514, various schemes of determining alane from a feature point in the lane determiner 340 may be used. Forexample, in lane determination operation S514, the lane determiner 340may detect the longest outline among lines represented by feature pointsextracted in the edge detector 330, and may determine the longestoutline as a lane.

The above description of the vehicle camera calibration apparatus isalso applicable to image processing operation S511, frame accumulationoperation S512, edge detection operation S513 and lane determinationoperation S514 in lane detection operation S510, and accordingly furtherdescription of image processing operation S511, frame accumulationoperation S512, edge detection operation S513 and lane determinationoperation S514 is not repeated herein.

Image conversion operation S520 may refer to an operation of extractinga feature point of a lane from a received image based on stored initialcamera information and stored initial external parameter information,and converting the received image into a top-view image, using the imageconverter 410 of the camera correction module 400.

As described above, when an external parameter is accurate and when aconversion to a top-view image is performed, lanes on the top-view imagemay be on the same straight line and may be vertically shown. However,when a position of a camera of the camera module 100 is changed by aphysical force, and when the received image is converted into a top-viewimage based on the initial camera information and the initial externalparameter information, the lanes on the top-view image may not exist ona straight line.

Thus, in image conversion operation S520, the received image may beconverted into the top-view image so that an external parameter of thechanged camera may be estimated in the next operation.

Equation calculation operation S530 may refer to an operation ofacquiring a lane equation and a lane width from the top-view imageacquired in image conversion operation 520, so that a new externalparameter may be easily estimated based on the lane equation and thelane width in the next operation.

In equation calculation operation S530, the equation calculator 420 ofthe camera correction module 400 may acquire the lane equation by linefitting of feature points in the top-view image acquired in the imageconverter 410. For the line fitting, a least mean square (LMS) algorithmmay be used. The LMS algorithm used for line fitting is merely anexample, and another algorithm may also be used.

The lane equation calculated in equation calculation operation S530 maybe an equation of a straight line, or a multi-dimensional equation. Anequation calculated when a lane equation is an equation of a straightline may be expressed as Equation 1 shown below.

y=ax+b  [Equation 1]

a: Slopes G_(l) and G_(r) of lanes that appear later

b: y-intercept of a lane

x, y: Coordinate points of a lane that are obtained in advance

In other words, in equation calculation operation S530, values of x andy in Equation 1 may be set from an image, and a purpose of an equationcalculation operation is to obtain a and b from the values. In Equation1, a denotes a slot of a lane that appears later, and b denotes ay-intercept of a lane. Lane equations, that is, a and b may be acquiredusing line fitting based on the values of x and y that are coordinatepoints of a lane that are obtained in advance.

Referring to FIG. 4, parameter estimation operation S540 may be anoperation of estimating a new external parameter based on the laneequation and the lane width calculated in the equation calculationoperation, and may include primary correction operation S541, secondarycorrection operation S542, tertiary correction operation S543 andstorage operation S544.

In primary correction operation S541, secondary correction operationS542 and tertiary correction operation S543 of parameter estimationoperation S540, lane detection operation S510, image conversionoperation S520 and equation calculation operation S530 are assumed to bethe same to calculate a lane equation and a lane width of a camera.

In other words, a process of detecting a lane using each of a frontcamera, a rear camera, a left camera and a right camera, performing aconversion to a top-view image and acquiring a lane equation and a lanewidth may be individually performed for each primary correctionoperation S541, secondary correction operation S542 and tertiarycorrection operation S543.

Although the process is represented as one process including lanedetection operation S510, image conversion operation S520 and equationcalculation operation S530 in an example embodiment, lane detectionoperation S510, image conversion operation S520 and equation calculationoperation S530 may be individually performed for each camera, actually,in primary correction operation S541, secondary correction operationS542 and tertiary correction operation S543.

In primary correction operation S541, whether correction information ofa pitch (Rx), a yaw (Ry) and a roll (Rz) of a front camera of a vehicleis accurate may be determined in operation S541 a 1, and whethercorrection information of a pitch (Rx), a yaw (Ry) and a roll (Rz) of arear camera of the vehicle is accurate may be determined in operationS541 b 1. When the correction information is accurate, the nextoperation may be performed. When the correction information isinaccurate, the correction information of the pitch (Rx), the yaw (Ry)and the roll (Rz) of the front camera may be calculated in operationS541 a 2, and the correction information of the pitch (Rx), the yaw (Ry)and the roll (Rz) of the rear camera may be calculated in operation S541b 2. Image conversion operation S520 corresponding to each of operationsS541 a 2 and S541 b 2 may be reperformed to perform a conversion to atop-view image, to acquire a lane equation and a lane width, and toreperform primary correction operation S541.

Operation S541 a 1 of determining whether the correction information ofthe pitch (Rx) of the front camera is accurate, and operation S541 b 1of determining whether the correction information of the pitch (Rx) ofthe rear camera is accurate in primary correction operation S541 aredescribed below.

FIG. 5 illustrates an example in which a pitch (Rx) of a front camera ofa vehicle and a pitch (Rx) of a rear camera of the vehicle are twisted.A left image of FIG. 5 illustrates an example in which the pitches (Rx)of the front camera and the rear camera are less than a ground truthvalue, that is, an actual reference value, and a right image of FIG. 5illustrates an example in which the pitches (Rx) of the front camera andthe rear camera are greater than the ground truth value.

When a slope of a lane is calculated based on a side lane, result valuesmay be expressed as + and −. In other words, when pitches (Rx) of thefront camera and the rear camera are different from each other in atop-view image, slopes of lanes may be expressed as + and −. When a“reciprocal (1/G_(l)) of a slope of a left lane=a reciprocal (1/G_(r))of a slope of a right lane” is satisfied, the pitches (Rx) of the frontcamera and the rear camera may be accurate external parameters.

Accordingly, operation S541 a 2 of calculating the correctioninformation of the pitch (Rx) of the front camera and operation S541 b 2of calculating the correction information of the pitch (Rx) of the rearcamera in primary correction operation S541 may be achieved byestimating new pitches (Rx) of the front camera and the rear camera asshown in Equation 2 below.

$\begin{matrix}{{{New}\mspace{14mu}{Rx}} = {{{PreR}x} - {\alpha\left( {\frac{1}{G_{1}} - \frac{1}{G_{r}}} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 2} \right\rbrack\end{matrix}$

New Rx: New pitches of the front camera and the rear camera

Pre Rx: Pitches of the front camera and the rear camera prior torepetition

α: Arbitrary constant value that may be designated by a user

G_(l): Slope of a left lane

G_(r): Slope of a right lane

In other words, in primary correction operation S541, the correctioninformation of the pitches (Rx) of the front camera and the rear cameramay be calculated by gradually modifying a new pitch (Rx) and repeatingthe process from image conversion operation S520, until“1/G_(l)=1/G_(r)” is satisfied, that is, until Equation 3 shown below issatisfied.

$\begin{matrix}{{\frac{1}{G_{1}} - \frac{1}{Gr}} = 0} & \left\lbrack {{Equation}\mspace{14mu} 3} \right\rbrack\end{matrix}$

G_(l): Slope of a left lane

G_(r): Slope of a right lane

When the process returns to image conversion operation S520, a top-viewimage may be generated again based on new pitches (Rx) of the frontcamera and the rear camera, operation S541 a 1 of determining whetherthe correction information of the pitch (Rx) of the front camera isaccurate and operation S541 b 1 of determining whether the correctioninformation of the pitch (Rx) of the rear camera is accurate may berepeated through equation calculation operation S530 of calculating alane equation in the top-view image.

Operation S541 a 1 of determining whether the correction information ofthe yaw (Ry) of the front camera is accurate, and operation S541 b 1 ofdetermining whether the correction information of the yaw (Ry) of therear camera is accurate in primary correction operation S541 aredescribed below.

FIG. 6 illustrates an example in which yaws (Ry) of a front camera and arear camera of a vehicle are twisted. A left image of FIG. 6 illustratesan example in which the yaws (Ry) of the front camera and the rearcamera are greater than a ground truth value, that is, an actualreference value, and a right image of FIG. 6 illustrates an example inwhich the yaws (Ry) of the front camera and the rear camera are lessthan the ground truth value. In this example, a width between a leftlane and a right lane shown in an upper portion of a vehicle image maybe different from a width between a left lane and a right lane shown ina lower portion of the vehicle image.

Accordingly, in operations S541 a 2, 542 a 2 and 541 b 2 of primarycorrection operation S541, the correction information of the yaws (Ry)of the front camera and the rear camera may be calculated by modifying anew yaw (Ry) and repeating the process from image conversion operationS520, until Equations 4 and 5 shown below are satisfied.

New Ry=PreRy+α(W ₁ −W _(r))  [Equation 4]

W _(l) −W _(r)=0  [Equation 5]

New Ry: New yaws of the front camera and the rear camera

Pre Ry: Yaws of the front camera and the rear camera prior to repetition

α: Arbitrary constant value that may be designated by a user

W_(l): Width of a left lane

W_(r): Width of a right lane

A process of returning to image conversion operation S520 and repeatingprimary correction operation S541 is the same as the above-describedprocess associated with the pitches (Rx) of the front camera and therear camera, and accordingly further description thereof is omittedherein.

Operation S541 a 1 of determining whether the correction information ofthe roll (Rz) of the front camera is accurate in primary correctionoperation S541 is described below.

FIG. 7 illustrates an example in which rolls (Rz) of a front camera anda rear camera of a vehicle are twisted. A left image of FIG. 7illustrates an example in which the rolls (Rz) of the front camera andthe rear camera are greater than a ground truth value, that is, anactual reference value, and a right image of FIG. 7 illustrates anexample in which the rolls (Rz) of the front camera and the rear cameraare less than the ground truth value. When the rolls (Rz) of the frontcamera and the rear camera are twisted, (1/G_(l)) and (1/G_(r)) may havethe same sign and the same value. When the correction information of theroll (Rz) is accurate, a value of “(1/G_(l))+(1/G_(r))” may be close to“0.”

Accordingly, in operations S541 a 2 and 542 b 2 of primary correctionoperation S541, the correction information of the rolls (Rz) of thefront camera and the rear camera may be calculated by modifying a newroll (Rz) and repeating the process from image conversion operationS520, until Equations 6 and 7 shown below are satisfied.

$\begin{matrix}{{{New}\mspace{20mu}{Rz}} = {{PreRz} - {\alpha\left( {\frac{1}{C_{1}} + \frac{1}{G_{r}}} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 6} \right\rbrack \\{{\frac{1}{G_{1}} - \frac{1}{Gr}} = 0} & \left\lbrack {{Equation}\mspace{14mu} 7} \right\rbrack\end{matrix}$

New Rz: New rolls of the front camera and the rear camera

Pre Ry: Rolls of the front camera and the rear camera prior torepetition

α: Arbitrary constant value that may be designated by a user

G_(l): Slope of a left lane

G_(r): Slope of a right lane

A process of returning to image conversion operation S520 and repeatingprimary correction operation S541 is the same as the above-describedprocess associated with the pitches (Rx) of the front camera and therear camera, and accordingly further description thereof is omittedherein.

In secondary correction operation S542, whether correction informationof a position translation (Tz) of the rear camera is accurate may bedetermined in operation S542 a 1. When the correction information isaccurate, the next operation may be performed. When the correctioninformation is inaccurate, the correction information of the positiontranslation (Tz) of the rear camera may be calculated in operation S542a 2 and image conversion operation S520 may be reperformed.

Operation S542 a 1 of determining whether the correction information ofthe rear camera is accurate in secondary correction operation S542 isdescribed below.

FIG. 8 illustrates an example in which a position translation (Tz) of arear camera of a vehicle is twisted. A left image of FIG. 8 illustratesan example in which position translations (Tz) of the front camera andthe rear camera are greater than a ground truth value, that is, anactual reference value, and a right image of FIG. 7 illustrates anexample in which the position translations (Tz) of the front camera andthe rear camera are less than the ground truth value. When the positiontranslation (Tz) of the rear camera is twisted, a width between lanes ofa road in front of a vehicle and a width between lanes of a road behindthe vehicle may be different from each other. When the positiontranslation (Tz) is accurate, the widths may be identical to each other.

Accordingly, in operation S542 a 2 of secondary correction operationS542, the correction information of the position translation (Tz) of therear camera may be calculated by modifying a new position translation(Tz) and repeating the process from the image conversion operation untilEquations 8 and 9 shown below are satisfied.

New Tz=PreTz−α(C _(f) −C _(r))  [Equation 8]

C _(f) −C _(r)=0  [Equation 9]

New Tz: New position translations of the front camera and the rearcamera

Pre Tz: Position translations of the front camera and the rear cameraprior to repetition

α: Arbitrary constant value that may be designated by a user

C_(f): Width between a left lane and a right lane of a front road

C_(r): Width between a left lane and a right lane of a rear road

A process of returning to image conversion operation S520 and repeatingsecondary correction operation S542 is the same as the above-describedprocess associated with the pitches (Rx) of the front camera and therear camera, and accordingly further description thereof is omittedherein.

In tertiary correction operation S543, whether correction information ofpitches (Rx), yaws (Ry), rolls (Rz) and position translations (Tz) of aleft camera and a right camera of the vehicle is accurate is determinedin operations S543 a 1 and 543 b 1. When the correction information isaccurate, the next operation may be performed. When the correctioninformation is inaccurate, the correction information of the pitches(Rx), the yaws (Ry), the rolls (Rz) and the position translations (Tz)of the left camera and the right camera may be calculated in operationsS543 a 2 and S543 b 2, and image conversion operation S520 may bereperformed.

Operations S543 a 1 and 543 b 1 of determining whether the correctioninformation of the pitches (Rx) of the left camera and the right camerais accurate in tertiary correction operation S543 are described below.

FIG. 9 illustrates an example in which pitches (Rx) of a left camera anda right camera of a vehicle are twisted. A left image of FIG. 9illustrates an example in which the pitches (Rx) of the left camera andthe right camera are greater than a ground truth value, that is, anactual reference value, and a right image of FIG. 9 illustrates anexample in which the pitches (Rx) of the left camera and the rightcamera are less than the ground truth value. When the pitches (Rx) ofthe left camera and the right camera are twisted, lanes located in afront side and a rear side of the vehicle, and lanes located in a leftside and a right side of the vehicle may not be on one straight line andnot be aligned. When the pitches (Rx) are accurate, the lanes may be onone straight line.

Accordingly, in operation S543 a 2 of tertiary correction operationS543, the correction information of the pitch (Rx) of the left cameramay be calculated by modifying a new pitch (Rx) and repeating theprocess from image conversion operation S520 until Equations 10 and 11shown below are satisfied.

$\begin{matrix}{{LeftCameraNewRx} = {{LeftCameraPreRx} - {\alpha\left( {\frac{\left( {x_{2} + x_{6}} \right)}{2} - \frac{\left( {x_{1} + x_{5}} \right)}{2}} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 10} \right\rbrack \\{\mspace{79mu}{{\frac{\left( {x_{2} + x_{6}} \right)}{2} - \frac{\left( {x_{1} + x_{5}} \right)}{2}} = 0}} & \left\lbrack {{Equation}\mspace{14mu} 11} \right\rbrack\end{matrix}$

New Rx: New pitch of the left camera

Pre Rx: Pitch of the left camera prior to repetition

α: Arbitrary constant value that may be designated by a user

x₁: Left x-coordinate of a left lane of a front road

x₂: Right x-coordinate of the left lane of the front road

x₅: Left x-coordinate of a left lane of a rear road

x₆: Right x-coordinate of the left lane of the rear road

A process of returning to image conversion operation S520 and repeatingtertiary correction operation S543 is the same as the above-describedprocess associated with the pitches (Rx) of the front camera and therear camera, and accordingly further description thereof is omittedherein.

Also, in operation S543 b 2 of tertiary correction operation S543, thecorrection information of the pitch (Rx) of the right camera may becalculated by modifying a new pitch (Rx) and repeating the process fromthe image conversion operation until Equations 12 and 13 shown below aresatisfied.

$\begin{matrix}{{RightCameraNewRx} = {{{Right}{CameraPreRx}} - {\alpha\left( {\frac{\left( {x_{3} + x_{7}} \right)}{2} - \frac{\left( {x_{4} + x_{8}} \right)}{2}} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 12} \right\rbrack \\{\mspace{79mu}{{\frac{\left( {x_{3} + x_{7}} \right)}{2} - \frac{\left( {x_{4} + x_{8}} \right)}{2}} = 0}} & \left\lbrack {{Equation}\mspace{14mu} 13} \right\rbrack\end{matrix}$

New Rx: New pitch of the right camera

Pre Rx: Pitch of the right camera prior to repetition

α: Arbitrary constant value that may be designated by a user

x₃: Left x-coordinate of a right lane of a front road

x₄: Right x-coordinate of the right lane of the front road

x₇: Left x-coordinate of a right lane of a rear road

x₈: Right x-coordinate of the right lane of the rear road

A process of returning to image conversion operation S520 and repeatingtertiary correction operation S543 is the same as the above-describedprocess associated with the pitches (Rx) of the front camera and therear camera, and accordingly further description thereof is omittedherein.

Operations S543 a 1 and S543 b 1 of determining whether the correctioninformation of the yaws (Ry) of the left camera and the right camera isaccurate in tertiary correction operation S543 are described below.

FIG. 10 illustrates an example in which yaws (Ry) of a left camera and aright camera of a vehicle are twisted. A left image of FIG. 10illustrates an example in which the yaws (Ry) of the left camera and theright camera are greater than a ground truth value, that is, an actualreference value, and a right image of FIG. 10 illustrates an example inwhich the yaws (Ry) of the left camera and the right camera are lessthan the ground truth value. When the yaws (Ry) of the left camera andthe right camera are accurate, (x₂₂−x₁₁) and (x₆₆−x₅₅) may be identicalto each other, or (x₄₄−x₃₃) and (x₈₈−x₇₇) may be identical to eachother.

Accordingly, in operation S543 a 2 of tertiary correction operationS543, the correction information of the yaw (Ry) of the left camera maybe calculated by modifying a new yaw (Ry) and repeating the process fromimage conversion operation S520 until Equations 14 and 15 shown beloware satisfied.

LeftCameraNewRy=LeftCameraPreRy−α((x ₂₂ −x ₁₁)−(x ₆₆−₅₅))  [Equation 14]

(x ₂₂ −x ₁₁)−(x ₆₆ −x ₅₅)=0  [Equation 15]

New Ry: New yaw of the left camera

Pre Ry: Yaw of the left camera prior to repetition

α: Arbitrary constant value that may be designated by a user

x₁₁: Left x-coordinate of a left lane of a front road

x₂₂: Right x-coordinate of the left lane of the front road

x₅₅: Left x-coordinate of a left lane of a rear road

x₆₆: Right x-coordinate of the left lane of the rear road

A process of returning to image conversion operation S520 and repeatingtertiary correction operation S543 is the same as the above-describedprocess associated with the pitches (Rx) of the front camera and therear camera, and accordingly further description thereof is omittedherein.

Also, in operation S543 b 2 of tertiary correction operation S543, thecorrection information of the yaw (Ry) of the right camera may becalculated by modifying a new yaw (Ry) and repeating the process fromthe image conversion operation until Equations 16 and 17 shown below aresatisfied.

RightCameraNewRy=RightCameraPreRy+α((x ₄₄ −x ₃₃)−(x ₈₈ −x₇₇))  [Equation 16]

(x ₄₄ −x ₃₃)−(x ₈₈ −x ₇₇)=0  [Equation 17]

New Ry: New yaw of the right camera

Pre Ry: Yaw of the right camera prior to repetition

α: Arbitrary constant value that may be designated by a user

x₃₃: Left x-coordinate of a right lane of a front road

x₄₄: Right x-coordinate of the right lane of the front road

x₇₇: Left x-coordinate of a right lane of a rear road

x₈₈: Right x-coordinate of the right lane of the rear road

A process of returning to image conversion operation S520 and repeatingtertiary correction operation S543 is the same as the above-describedprocess associated with the pitches (Rx) of the front camera and therear camera, and accordingly further description thereof is omittedherein.

Operations S543 a 1 and S543 b 1 of determining whether the correctioninformation of the rolls (Rz) of the left camera and the right camera intertiary correction operation S543 are described below.

FIG. 11 illustrates an example in which rolls (Rz) of a left camera anda right camera of a vehicle are twisted. A left image of FIG. 11illustrates an example in which the rolls (Rz) of the left camera andthe right camera are greater than a ground truth value, that is, anactual reference value, and a right image of FIG. 11 illustrates anexample in which the rolls (Rz) of the left camera and the right cameraare less than the ground truth value. When the rolls (Rz) of the leftcamera and the right camera are twisted, 1/G_(l), and 1/G, may havegreater values. When the rolls (Rz) are accurate, 1/G_(l) and 1/G, mayhave values close to “0.”

Accordingly, in operation S543 a 2 of tertiary correction operationS543, the correction information of the roll (Rz) of the left camera maybe calculated by modifying a new roll (Rz) and repeating the processfrom the image conversion operation until Equations 18 and 19 shownbelow are satisfied.

$\begin{matrix}{{leftCameraNewRz} = {{leftCameraPreRz} - {\alpha\left( \frac{1}{G_{1}} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 18} \right\rbrack \\{\frac{1}{G_{1}} = 0} & \left\lbrack {{Equation}\mspace{14mu} 19} \right\rbrack\end{matrix}$

New Rz: New roll of the left camera

Pre Rz: Roll of the left camera prior to repetition

α: Arbitrary constant value that may be designated by a user

G_(l): Slope of a left lane

A process of returning to image conversion operation S520 and repeatingtertiary correction operation S543 is the same as the above-describedprocess associated with the pitches (Rx) of the front camera and therear camera, and accordingly further description thereof is omittedherein.

Also, in operation S543 b 2 of tertiary correction operation S543, thecorrection information of the roll (Rz) of the right camera may becalculated by modifying a new roll (Rz) and repeating the process fromthe image conversion operation until Equations 20 and 21 shown below aresatisfied.

$\begin{matrix}{{RightCameraNewRz} = {{RightCameraPreRz} - {\alpha\left( \frac{1}{G_{r}} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 20} \right\rbrack \\{\mspace{76mu}{\frac{1}{G_{r}} = 0}} & \left\lbrack {{Equation}\mspace{14mu} 21} \right\rbrack\end{matrix}$

New Rz: New roll of the right camera

Pre Rz: Roll of the right camera prior to repetition

α: Arbitrary constant value that may be designated by a user

G_(r): Slope of a right lane

A process of returning to image conversion operation S520 and repeatingtertiary correction operation S543 is the same as the above-describedprocess associated with the pitches (Rx) of the front camera and therear camera, and accordingly further description thereof is omittedherein.

Operations S543 a 1 and S543 b 1 of determining whether the correctioninformation of the position translations (Tz) of the left camera and theright camera is accurate in tertiary correction operation S543 aredescribed below.

FIG. 12 illustrates an example in which position translations (Tz) of aleft camera and a right camera of a vehicle are twisted. A left image ofFIG. 12 illustrates an example in which the position translations (Tz)of the left camera and the right camera are greater than a ground truthvalue, that is, an actual reference value, and a right image of FIG. 12illustrates an example in which the position translations (Tz) of theleft camera and the right camera are less than the ground truth value.When the position translations (Tz) of the left camera and the rightcamera are accurate, widths between lanes on the front side and the rearside of the vehicle may be identical to widths between lanes on the leftside and the right side of the vehicle.

Accordingly, in operation S543 a 2 of tertiary correction operationS543, the correction information of the position translation (Tz) of theleft camera may be calculated by modifying a new position translation(Tz) and repeating the process from the image conversion operation untilEquations 22 and 23 shown below are satisfied.

$\begin{matrix}{{LeftCameraNewTz} = {{LeftCameraPreTz} - {\alpha\left( {\frac{\left( {x_{222} - x_{111}} \right) + \left( {x_{10} - x_{999}} \right)}{2} - \left( {x_{666} - x_{555}} \right)} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 22} \right\rbrack \\{\mspace{70mu}{{\frac{\left( {x_{222} - x_{111}} \right) + \left( {x_{10} - x_{999}} \right)}{2} - \left( {x_{666} - x_{555}} \right)} = 0}} & \left\lbrack {{Equation}\mspace{14mu} 23} \right\rbrack\end{matrix}$

New Tz: New position translation of the left camera

Pre Tz: Position translation of the left camera prior to repetition

α: Arbitrary constant value that may be designated by a user

x₁₁₁: Left x-coordinate of a left lane of a front road

x₂₂₂: Right x-coordinate of the left lane of the front road

x₅₅₅: Left x-coordinate of a left lane

x₆₆₆: Right x-coordinate of the left lane

x₉₉₉: Left x-coordinate of a left lane of a rear road

x₁₀: Right x-coordinate of the left lane of the rear road A process ofreturning to image conversion operation S520 and repeating tertiarycorrection operation S543 is the same as the above-described processassociated with the pitches (Rx) of the front camera and the rearcamera, and accordingly further description thereof is omitted herein.

Also, in operation S543 b 2 of tertiary correction operation S543, thecorrection information of the position translation (Tz) of the rightcamera may be calculated by modifying a new position translation (Tz)and repeating the process from the image conversion operation untilEquations 24 and 25 shown below are satisfied.

$\begin{matrix}{{RightCameraNewTz} = {{ReftCameraPreTz} + {\alpha\left( {\frac{\left( {x_{444} - x_{333}} \right) + \left( {x_{10} - x_{999}} \right)}{2} - \left( {x_{888} - x_{777}} \right)} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 24} \right\rbrack \\{\mspace{70mu}{{\frac{\left( {x_{444} - x_{333}} \right) + \left( {x_{12} - x_{11}} \right)}{2} - \left( {x_{888} - x_{777}} \right)} = 0}} & \left\lbrack {{Equation}\mspace{14mu} 25} \right\rbrack\end{matrix}$

New Tz: New position translation of the right camera

Pre Tz: Position translation of the right camera prior to repetition

α: Arbitrary constant value that may be designated by a user

x₃₃₃: Left x-coordinate of a right lane of a front road

x₄₄₄: Right x-coordinate of the right lane of the front road

x₇₇₇: Left x-coordinate of a right lane

x₈₈₈: Right x-coordinate of the right lane

x₁₁: Left x-coordinate of a right lane of a rear road

x₁₂: Right x-coordinate of the right lane of the rear road

A process of returning to image conversion operation S520 and repeatingtertiary correction operation S543 is the same as the above-describedprocess associated with the pitches (Rx) of the front camera and therear camera, and accordingly further description thereof is omittedherein.

Storage operation S544 may be an operation of repeatedly performingprimary correction operation S541, secondary correction operation S542and tertiary correction operation S543 until accurate correction isrealized, and of storing a finally calculated external parameter.

Image correction operation S550 may be an operation of correcting animage based on a new external parameter estimated in parameterestimation operation S540, and the corrected image may be transmitted tothe storage 220 of the input/output module 200.

As a result, in the vehicle camera calibration apparatus and methodaccording to an example embodiment, it is possible to automaticallycalibrate an external parameter even when a vehicle is travelling.

As described above, according to example embodiments, a cameracalibration may be performed by receiving images representing atravelling road of a vehicle from a plurality of cameras installed inthe vehicle, by detecting a lane from the images, and by correcting anexternal parameter of a camera installed in the vehicle based on thedetected lane. Thus, it is possible to quickly and simply calibrate acamera without a need to stop the vehicle.

While this disclosure includes specific examples, it will be apparent toone of ordinary skill in the art that various changes in form anddetails may be made in these examples without departing from the spiritand scope of the claims and their equivalents. The examples describedherein are to be considered in a descriptive sense only, and not forpurposes of limitation. Descriptions of features or aspects in eachexample are to be considered as being applicable to similar features oraspects in other examples. Suitable results may be achieved if thedescribed techniques are performed in a different order, and/or ifcomponents in a described system, architecture, device, or circuit arecombined in a different manner, and/or replaced or supplemented by othercomponents or their equivalents. Therefore, the scope of the disclosureis defined not by the detailed description, but by the claims and theirequivalents, and all variations within the scope of the claims and theirequivalents are to be construed as being included in the disclosure.

1. A vehicle camera calibration apparatus comprising: a camera moduleconfigured to acquire an image representing a road from a plurality ofcameras installed in a vehicle; an input/output module configured toreceive, as an input, the acquired image from the camera module, oroutput a corrected image; a lane detection module configured to detect alane and extract a feature point of the lane from an image received fromthe input/output module; and a camera correction module configured tocalculate a lane equation and a lane width based on initial camerainformation and external parameter information in the image receivedfrom the input/output module, to estimate a new external parameter, andto correct the image.
 2. The vehicle camera calibration apparatus ofclaim 1, wherein the input/output module comprises: an imageinputter/outputter configured to receive, as an input, an image from thecamera module, or to output a corrected image to another module; and astorage configured to receive the corrected image and changed externalparameter information from the camera correction module, and to storethe corrected image and the changed external parameter information. 3.The vehicle camera calibration apparatus of claim 1, wherein the lanedetection module comprises: an image processor configured to removenoise from the image received from the input/output module; a frameaccumulator configured to accumulate images processed by the imageprocessor as consecutive frames, when a lane is marked with a dashedline; an edge detector configured to detect an edge and to extract afeature point from a cumulative image including the images accumulatedby the frame accumulator; and a lane determiner configured to determinea lane from the feature point extracted by the edge detector.
 4. Thevehicle camera calibration apparatus of claim 1, wherein the cameracorrection module comprises: an image converter configured to extract afeature point of a lane from a received image based on initial camerainformation and external parameter information, and to convert thereceived image into a top-view image; an equation calculator configuredto acquire a lane equation from the top-view image acquired by the imageconverter; a parameter estimator configured to estimate a new externalparameter using the lane equation calculated by the equation calculator;and a corrector configured to correct an image using the new externalparameter estimated by the parameter estimator.
 5. A vehicle cameracalibration method comprising: a lane detection operation of receiving,as an input, an acquired image from a camera module, and detecting alane; an image conversion operation of extracting a feature point of alane from a received image based on initial camera information andexternal parameter information, and converting the received image into atop-view image; an equation calculation operation of acquiring a laneequation and a lane width from the top-view image acquired in the imageconversion operation; a parameter estimation operation of estimating anew external parameter using the lane equation and the lane widthcalculated in the equation calculation operation; and an imagecorrection operation of correcting an image using the new externalparameter estimated in the parameter estimation operation.
 6. Thevehicle camera calibration method of claim 5, wherein the lane detectionoperation comprises: an image processing operation of removing noisefrom an image received from an input/output module; a frame accumulationoperation of accumulating images processed in the image processingoperation as consecutive frames when a lane is marked with a dashed linein an image processed in the image processing operation, and performinga next operation when the lane is marked with a straight line; an edgedetection operation of detecting an edge and extracting a feature pointfrom a cumulative image including the images accumulated in the frameaccumulation operation; and a lane determination operation ofdetermining a lane from the feature point extracted in the edgedetection operation.
 7. The vehicle camera calibration method of claim6, wherein the image processing operation comprises performing analgorithm of softening a boundary of an object in an image to removenoise.
 8. The vehicle camera calibration method of claim 6, wherein theimage processing operation comprises removing noise by changing a sizeof an image to prevent a region corresponding to a split lane in anacquired image from being detected as an edge.
 9. The vehicle cameracalibration method of claim 6, wherein the frame accumulation operationcomprises comparing all pixel values of the cumulative image and storinga highest pixel value in a new image so that a dashed-line lane in anacquired image is viewed to be identical to a solid-line lane.
 10. Thevehicle camera calibration method of claim 6, wherein the edge detectionoperation comprises detecting an edge using an algorithm and extractingthe edge and central coordinates of the edge as feature points.
 11. Thevehicle camera calibration method of claim 6, wherein the lanedetermination operation comprises detecting a longest outline amonglines represented by feature points extracted in the edge detectionoperation and determining the longest outline as a lane.
 12. The vehiclecamera calibration method of claim 5, wherein the lane equationcalculated in the equation calculation operation is expressed as anequation of a straight line, or a multi-dimensional equation.
 13. Thevehicle camera calibration method of claim 5, wherein the parameterestimation operation comprises: a primary correction operation ofdetermining whether correction information of a pitch (Rx), a yaw (Ry)and a roll (Rz) of a front camera of a vehicle, and correctioninformation of a pitch (Rx), a yaw (Ry) and a roll (Rz) of a rear cameraof the vehicle are accurate, of performing a next operation when thecorrection information is accurate, and of calculating the correctioninformation of the pitch (Rx), the yaw (Ry) and the roll (Rz) of thefront camera and the correction information of the pitch (Rx), the yaw(Ry) and the roll (Rz) of the rear camera and returning to the imageconversion operation when the correction information is inaccurate; asecondary correction operation of determining whether correctioninformation of a position translation (Tz) of a rear camera of a vehicleis accurate, of performing a next operation when the correctioninformation is accurate, and of calculating the correction informationof the position translation (Tz) of the rear camera and returning to theimage conversion operation when the correction information isinaccurate; a tertiary correction operation of determining whethercorrection information of a pitch (Rx), a yaw (Ry), a roll (Rz) and aposition translation (Tz) of each of a left camera and a right camera ofa vehicle is accurate, of performing a next operation when thecorrection information is accurate, and of calculating the correctioninformation of the pitch (Rx), the yaw (Ry), the roll (Rz) and theposition translation (Tz) of each of the left camera and the rightcamera and returning to the image conversion operation when thecorrection information is inaccurate; and a storage operation of storingexternal parameters calculated in the primary correction operationthrough the tertiary correction operation.
 14. The vehicle cameracalibration method of claim 13, wherein the correction information ofthe pitch (Rx) of each of the front camera and the rear camera in theprimary correction operation is calculated by modifying a new pitch (Rx)and repeating a process from the image conversion operation untilEquation 1 shown below is satisfied. $\begin{matrix}{{\frac{1}{G_{1}} - \frac{1}{G_{r}}} = 0} & \left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack\end{matrix}$ wherein G_(l) denotes a slope of a left lane, and G_(r)denotes a slope of a right lane.
 15. The vehicle camera calibrationmethod of claim 13, wherein the correction information of the yaw (Ry)of each of the front camera and the rear camera in the primarycorrection operation and the secondary correction operation iscalculated by modifying a new yaw (Ry) and repeating a process from theimage conversion operation until Equation 2 shown below is satisfied.W ₁ −W _(r)=0  [Equation 2] wherein W_(l) denotes a width of a leftlane, and W_(r) denotes a width of a right lane.
 16. The vehicle cameracalibration method of claim 13, wherein the correction information ofthe roll (Rz) of each of the front camera and the rear camera in theprimary correction operation is calculated by modifying a new roll (Rz)and repeating a process from the image conversion operation untilEquation 3 shown below is satisfied. $\begin{matrix}{{\frac{1}{G_{1}} - \frac{1}{G_{r}}} = 0} & \left\lbrack {{Equation}\mspace{14mu} 3} \right\rbrack\end{matrix}$ wherein G_(l) denotes a slope of a left lane, and G_(r)denotes a slope of a right lane.
 17. The vehicle camera calibrationmethod of claim 13, wherein the correction information of the positiontranslation (Tz) of the rear camera in the secondary correctionoperation is calculated by modifying a new position translation (Tz) andrepeating a process from the image conversion operation until Equation 4shown below is satisfied.C _(f) −C _(r)=0  [Equation 4] wherein C_(f) denotes a width between aleft lane and a right lane in a front road, and C_(r) denotes a widthbetween a left lane and a right lane in a rear road.
 18. The vehiclecamera calibration method of claim 13, wherein the correctioninformation of the pitch (Rx) of the left camera in the tertiarycorrection operation is calculated by modifying a new pitch (Rx) andrepeating a process from the image conversion operation until Equation 5shown below is satisfied, and the correction information of the pitch(Rx) of the right camera in the tertiary correction operation iscalculated by modifying a new pitch (Rx) and repeating a process fromthe image conversion operation until Equation 6 shown below issatisfied. $\begin{matrix}{{\frac{\left( {x_{2} + x_{6}} \right)}{2} - \frac{\left( {x_{1} + x_{5}} \right)}{2}} = 0} & \left\lbrack {{Equation}\mspace{14mu} 5} \right\rbrack \\{{\frac{\left( {x_{3} + x_{7}} \right)}{2} - \frac{\left( {x_{4} + x_{8}} \right)}{2}} = 0} & \left\lbrack {{Equation}\mspace{14mu} 6} \right\rbrack\end{matrix}$ wherein x₁ denotes a left x-coordinate of a left lane of afront road, x₂ denotes a right x-coordinate of the left lane of thefront road, x₅ denotes a left x-coordinate of a left lane of a rearroad, x₆ denotes a right x-coordinate of the left lane of the rear road,x₃ denotes a left x-coordinate of a right lane of the front road, x₄denotes a right x-coordinate of the right lane of the front road, x₇denotes a left x-coordinate of a right lane of the rear road, and x₈denotes a right x-coordinate of the right lane of the rear road.
 19. Thevehicle camera calibration method of claim 13, wherein the correctioninformation of the yaw (Ry) of the left camera in the tertiarycorrection operation is calculated by modifying a new yaw (Ry) andrepeating a process from the image conversion operation until Equation 7shown below is satisfied, and the correction information of the yaw (Ry)of the right camera in the tertiary correction operation is calculatedby modifying a new yaw (Ry) and repeating a process from the imageconversion operation until Equation 8 shown below is satisfied.(x ₂₂ −x ₁₁)−(x ₆₆ −x ₅₅)=0[Equation 7](x ₄₄ −x ₃₃)−(x ₈₈ −x ₇₇)=0  [Equation 8] wherein x₁₁ denotes a leftx-coordinate of a left lane of a front road, x₂₂ denotes a rightx-coordinate of the left lane of the front road, x₅₅ denotes a leftx-coordinate of a left lane of a rear road, x₆₆ denotes a rightx-coordinate of the left lane of the rear road, x₃₃ denotes a leftx-coordinate of a right lane of the front road, x₄₄ denotes a rightx-coordinate of the right lane of the front road, x₇₇ denotes a leftx-coordinate of a right lane of the rear road, and x₈₈ denotes a rightx-coordinate of the right lane of the rear road.
 20. The vehicle cameracalibration method of claim 13, wherein the correction information ofthe roll (Rz) of the left camera in the tertiary correction operation iscalculated by modifying a new roll (Rz) and repeating a process from theimage conversion operation until Equation 9 shown below is satisfied,and the correction information of the roll (Rz) of the right camera inthe tertiary correction operation is calculated by modifying a new roll(Rz) and repeating a process from the image conversion operation untilEquation 10 shown below is satisfied. $\begin{matrix}{\frac{1}{G_{1}} = 0} & \left\lbrack {{Equation}\mspace{14mu} 9} \right\rbrack \\{\frac{1}{G_{r}} = 0} & \left\lbrack {{Equation}\mspace{14mu} 10} \right\rbrack\end{matrix}$ wherein G_(l) denotes a slope of a left lane, and G_(r)denotes a slope of a right lane.
 21. The vehicle camera calibrationmethod of claim 13, wherein the correction information of the positiontranslation (Tz) of the left camera in the tertiary correction operationis calculated by modifying a new position translation (Tz) and repeatinga process from the image conversion operation until Equation 11 shownbelow is satisfied, and the correction information of the positiontranslation (Tz) of the right camera in the tertiary correctionoperation is calculated by modifying a new position translation (Tz) andrepeating a process from the image conversion operation until Equation12 shown below is satisfied. $\begin{matrix}{{\frac{\left( {x_{222} - x_{111}} \right) + \left( {x_{10} - x_{999}} \right)}{2} - \left( {x_{666} - x_{555}} \right)} = 0} & \left\lbrack {{Equation}\mspace{14mu} 11} \right\rbrack \\{{\frac{\left( {x_{444} - x_{333}} \right) + \left( {x_{12} - x_{11}} \right)}{2} - \left( {x_{888} - x_{777}} \right)} = 0} & \left\lbrack {{Equation}\mspace{14mu} 12} \right\rbrack\end{matrix}$ wherein x₁₁₁ denotes a left x-coordinate of a left lane ofa front road, x₂₂₂ denotes a right x-coordinate of the left lane of thefront road, x₅₅₅ denotes a left x-coordinate of a left lane, x₆₆₆denotes a right x-coordinate of the left lane, x₉₉₉ denotes a leftx-coordinate of a left lane of the rear road, x₁₀ denotes a rightx-coordinate of the left lane of the rear road, x₃₃₃ denotes a leftx-coordinate of a right lane of a front road, x₄₄₄ denotes a rightx-coordinate of a right lane of a front road, x₇₇₇ denotes a leftx-coordinate of a right lane, x₈₈₈ denotes a right x-coordinate of aright lane, x₁₁ denotes a left x-coordinate of a right lane of the rearroad, and x₁₂ denotes a right x-coordinate of the right lane of the rearroad.